Network memory pools for packet destinations and virtual machines

ABSTRACT

A method for processing a packet that includes receiving a packet for a target, classifying the packet, and sending the packet to a receive ring based on the classification. The method also includes obtaining an identifier (ID) associated with the target based on the classification, and sending a request for virtual memory that includes the ID. Furthermore, the method includes determining, using the ID, whether the target has exceeded a virtual memory allocation associated with the target. In addition, the method includes allocating the virtual memory, storing the packet in the virtual memory, and updating the virtual memory allocation associated with the target to reflect the allocation of the virtual memory, all if the target does not exceed the virtual memory allocation. The method further includes waiting until the target is not exceeding the virtual memory allocation if the target exceeds the virtual memory allocation.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application contains subject matter that may be related tothe subject matter in the following U.S. applications filed on Apr. 22,2005, and assigned to the assignee of the present application: “Methodand Apparatus for Managing and Accounting for Bandwidth UtilizationWithin A Computing System” with U.S. application Ser. No. 11/112,367(Attorney Docket No. 03226/643001; SUN050681); “Method and Apparatus forConsolidating Available Computing Resources on Different ComputingDevices” with U.S. application Ser. No. 11/112,368 (Attorney Docket No.03226/644001; SUN050682); “Assigning Higher Priority to TransactionsBased on Subscription Level” with U.S. application Ser. No. 11/112,947(Attorney Docket No. 03226/645001; SUN050589); “Method and Apparatus forDynamically Isolating Affected Services Under Denial of Service Attack”with U.S. application Ser. No. 11/112,158 (Attorney Docket No.03226/646001; SUN050587); “Method and Apparatus for Improving UserExperience for Legitimate Traffic of a Service Impacted by Denial ofService Attack” with U.S. application Ser. No. 11/112,629 (AttorneyDocket No. 03226/647001; SUN050590); “Method and Apparatus for LimitingDenial of Service Attack by Limiting Traffic for Hosts” with U.S.application Ser. No. 11/112,328 (Attorney Docket No. 03226/648001;SUN050591); “Hardware-Based Network Interface Per-Ring ResourceAccounting” with U.S. application Ser. No. 11/112,222 (Attorney DocketNo. 03226/649001; SUN050593); “Dynamic Hardware Classification EngineUpdating for a Network Interface” with U.S. application Ser. No.11/112,934 (Attorney Docket No. 03226/650001; SUN050592); “NetworkInterface Card Resource Mapping to Virtual Network Interface Cards” withU.S. application Ser. No. 11/112,063 (Attorney Docket No. 03226/651001;SUN050588); “Network Interface Decryption and Classification Technique”with U.S. application Ser. No. 11/112,436 (Attorney Docket No.03226/652001; SUN050596); “Method and Apparatus for Enforcing ResourceUtilization of a Container” with U.S. application Ser. No. 11/112,910(Attorney Docket No. 03226/653001; SUN050595); “Method and Apparatus forEnforcing Packet Destination Specific Priority Using Threads” with U.S.application Ser. No. 11/112,584 (Attorney Docket No. 03226/654001;SUN050597); “Method and Apparatus for Processing Network TrafficAssociated with Specific Protocols” with U.S. application Ser. No.11/112,228 (Attorney Docket No. 03226/655001; SUN050598).

The present application contains subject matter that may be related tothe subject matter in the following U.S. applications filed on Oct. 21,2005, and assigned to the assignee of the present application: “Methodand Apparatus for Defending Against Denial of Service Attacks” with U.S.application Ser. No. 11/255,366 (Attorney Docket No. 03226/688001;SUN050966); “Router Based Defense Against Denial of Service AttacksUsing Dynamic Feedback from Attacked Host” with U.S. application Ser.No. 11/256,254 (Attorney Docket No. 03226/689001; SUN050969); and“Method and Apparatus for Monitoring Packets at High Data Rates” withU.S. application Ser. No. 11/226,790 (Attorney Docket No. 03226/690001;SUN050972).

The present application contains subject matter that may be related tothe subject matter in the following U.S. applications filed on Jun. 30,2006, and assigned to the assignee of the present application: “NetworkInterface Card Virtualization Based On Hardware Resources and SoftwareRings” with U.S. Application Serial No. TBD (Attorney Docket No.03226/870001; SUN061020); “Method and System for Controlling VirtualMachine Bandwidth” with U.S. Application Serial No. TBD (Attorney DocketNo. 03226/871001; SUN061021); “Virtual Switch” with U.S. ApplicationSerial No. TBD (Attorney Docket No. 03226/873001; SUN061023); “Systemand Method for Virtual Network Interface Cards Based on InternetProtocol Addresses” with U.S. Application Serial No. TBD (AttorneyDocket No. 03226/874001; SUN061024); “Virtual Network Interface CardLoopback Fastpath” with U.S. Application Serial No. TBD (Attorney DocketNo. 03226/876001; SUN061027); “Bridging Network Components” with U.S.Application Serial No. TBD (Attorney Docket No. 03226/877001;SUN061028); “Reflecting the Bandwidth Assigned to a Virtual NetworkInterface Card Through Its Link Speed” with U.S. Application Serial No.TBD (Attorney Docket No. 03226/878001; SUN061029); “Method and Apparatusfor Containing a Denial of Service Attack Using Hardware Resources on aVirtual Network Interface Card” with U.S. Application Serial No. TBD(Attorney Docket No. 03226/879001; SUN061033); “Virtual NetworkInterface Cards with VLAN Functionality” with U.S. Application SerialNo. TBD (Attorney Docket No. 03226/882001; SUN061037); “Method andApparatus for Dynamic Assignment of Network Interface Card Resources”with U.S. Application Serial No. TBD (Attorney Docket No. 03226/883001;SUN061038); “Generalized Serialization Queue Framework for ProtocolProcessing” with U.S. Application Serial No. TBD (Attorney Docket No.03226/884001; SUN061039); “Serialization Queue Framework forTransmitting Packets” with U.S. Application Serial No. TBD (AttorneyDocket No. 03226/885001; SUN061040).

The present application contains subject matter that may be related tothe subject matter in the following U.S. applications filed on Jul. 20,2006, and assigned to the assignee of the present application: “LowImpact Network Debugging” with U.S. Application Serial No. TBD (AttorneyDocket No. 03226/829001; SUN060545); “Reflecting Bandwidth and Priorityin Network Attached Storage I/O” with U.S. Application Serial No. TBD(Attorney Docket No. 03226/830001; SUN060587); “Priority and BandwidthSpecification at Mount Time of NAS Device Volume” with U.S. ApplicationSerial No. TBD (Attorney Docket No. 03226/831001; SUN060588); “NotifyingNetwork Applications of Receive Overflow Conditions” with U.S.Application Serial No. TBD (Attorney Docket No. 03226/869001;SUN060913); “Host Operating System Bypass for Packets Destined for aVirtual Machine” with U.S. Application Serial No. TBD (Attorney DocketNo. 03226/872001; SUN061022); “Multi-Level Packet Classification” withU.S. Application Serial No. TBD (Attorney Docket No. 03226/875001;SUN061026); “Method and System for Automatically Reflecting HardwareResource Allocation Modifications” with U.S. Application Serial No. TBD(Attorney Docket No. 03226/881001; SUN061036); “Multiple Virtual NetworkStack Instances Using Virtual Network Interface Cards” with U.S.Application Serial No. TBD (Attorney Docket No. 03226/888001;SUN061041); “Method and System for Network Configuration for Containers”with U.S. Application Serial No. TBD (Attorney Docket No. 03226/889001;SUN061044); “Method and System for Network Configuration for VirtualMachines” with U.S. Application Serial No. TBD (Attorney Docket No.03226/893001; SUN061171); “Multiple Virtual Network Stack Instances”with U.S. Application Serial No. TBD (Attorney Docket No. 03226/896001;SUN061198); and “Shared and Separate Network Stack Instances” with U.S.Application Serial No. TBD (Attorney Docket No. 03226/898001;SUN061200).

BACKGROUND

Network traffic is transmitted over a network, such as the Internet,from a sending system (e.g., a computer system) to a receiving system(e.g., a computer system) via a network interface card (NIC). The NIC isa piece of hardware found in a typical computer system that includesfunctionality to send and receive network traffic. Typically, networktraffic is transmitted in the form of packets, where each packetincludes a header and a payload. The header contains informationregarding the source address, destination address, size, transportprotocol used to transmit the packet, and various other identificationinformation associated with the packet. The payload contains the actualdata to be transmitted from the network to the receiving system.

Each of the packets sent between the sending system and receiving systemis typically associated with a connection. The connection ensures thatpackets from a given process on the sending system reach the appropriateprocess on the receiving system. Packets received by the receivingsystem (via a NIC associated with the receiving system) are analyzed bya classifier to determine the connection associated with the packet.

Typically, the classifier includes a connection data structure thatincludes information about active connections on the receiving system.The connection data structure may include the following informationabout each active connection: (i) the queue associated with theconnection and (ii) information necessary to process the packets on thequeue associated with the connection. Depending on the implementation,the connection data structure may include additional information abouteach active connection. Such queues are typically implemented asfirst-in first-out (FIFO) queues and are bound to a specific centralprocessing unit (CPU) on the receiving computer system. Thus, allpackets for a given connection are placed in the same queue and areprocessed by the same CPU. In addition, each queue is typicallyconfigured to support multiple connections.

Once the classifier determines the connection associated with thepackets, the packets are forwarded to a temporary data structure (e.g.,a receive ring on the NIC) and an interrupt is issued to the CPUassociated with the queue. In response to the interrupt, a threadassociated with the CPU (to which the serialization queue is bound)retrieves the packets from the temporary data structure and places themin the appropriate queue. Once packets are placed in the queue, thosepackets are processed in due course. In some implementations, the queuesare implemented such that only one thread is allowed to access a givenqueue at any given time.

SUMMARY

In general, in one aspect, the invention relates to a method forprocessing a packet that includes receiving a packet for a first targeton a host, classifying the packet, and sending the packet to a receivering based on the classification. The method also includes obtaining anidentifier (ID) based on the classification, wherein the ID isassociated with the first target, and sending a request for virtualmemory on the host, wherein the request includes the ID. Furthermore,the method includes determining, using the ID, whether the first targethas exceeded a virtual memory allocation associated with the firsttarget. In addition, the method includes allocating the virtual memory,storing the packet in the virtual memory on the host, and updating thevirtual memory allocation associated with the first target to reflectthe allocation of the virtual memory, all if the first target does notexceed the virtual memory allocation. The method further includeswaiting until the first target is not exceeding the virtual memoryallocation if the first target exceeds the virtual memory allocation.

In general, on one aspect, the invention relates to a system thatincludes a network interface card (NIC) and a host operatively connectedto the NIC. The NIC is configured to: receive a packet for a firsttarget on a host; classify the packet; send the packet to a receive ringbased on the classification; obtain an ID based on the classification,wherein the ID is associated with the first target; and send a requestfor virtual memory on the host, wherein the request comprises the ID.The host is configured to determine, using the ID, whether the firsttarget has exceeded a virtual memory allocation associated with thefirst target. The host is further configured to: allocate the virtualmemory; obtain the packet from the receive ring; store the packet in thevirtual memory on the host; and update the virtual memory allocationassociated with the first target to reflect the allocation of thevirtual memory, all if the first target does not exceed the virtualmemory allocation. In addition, the host is configured to wait until thefirst target is not exceeding the virtual memory allocation if the firsttarget exceeds the virtual memory allocation.

In general, in one aspect, the invention relates to a computer readablemedium including instructions for a method for processing a packet. Themethod includes receiving a packet for a first target on a host,classifying the packet, and sending the packet to a receive ring basedon the classification. The method also includes obtaining an identifier(ID) based on the classification, wherein the ID is associated with thefirst target, and sending a request for virtual memory on the host,wherein the request comprises the ID. Furthermore, the method includesdetermining, using the ID, whether the first target has exceeded avirtual memory allocation associated with the first target. In addition,the method includes allocating the virtual memory, storing the packet inthe virtual memory on the host, and updating the virtual memoryallocation associated with the first target to reflect the allocation ofthe virtual memory, all if the first target does not exceed the virtualmemory allocation. The method further includes waiting until the firsttarget is not exceeding the virtual memory allocation if the firsttarget exceeds the virtual memory allocation.

Other aspects of the invention will be apparent from the followingdescription and the appended claims.

BRIEF DESCRIPTION OF DRAWINGS

FIGS. 1-2 shows schematic diagrams in accordance with one or moreembodiments of the invention.

FIGS. 3-4 show flow diagrams in accordance with one or more embodimentsof the invention.

FIG. 5 shows a computer system in accordance with one or moreembodiments of the invention.

DETAILED DESCRIPTION

Specific embodiments of the invention will now be described in detailwith reference to the accompanying figures. Like elements in the variousfigures are denoted by like reference numerals for consistency.

In the following detailed description of embodiments of the invention,numerous specific details are set forth in order to provide a morethorough understanding of the invention. However, it will be apparent toone of ordinary skill in the art that the invention may be practicedwithout these specific details. In other instances, well-known featureshave not been described in detail to avoid unnecessarily complicatingthe description.

In general, embodiments of the invention provide a method and apparatusto allocate virtual memory for network packets. Packets received by anetwork interface card (NIC) are classified and placed in receive ringsin the NIC before entering the host. Because packets are firstclassified, a virtual memory limit may be enforced such that incomingpackets are allowed into the host only when enough virtual memory ispresent to store them. In addition, outgoing packets are transmittedonly when enough virtual memory is available to process the packets andsend the packets to transmit rings on the NIC.

In one or more embodiments of the invention, virtual memory is allocatedbased on the priority of packet destinations or virtual machinesassociated with the packets. In addition, shortages in virtual memorymay be handled by transferring virtual memory from lower priority packetdestinations or virtual machines to higher priority packet destinationsor virtual machines.

FIG. 1 shows a schematic diagram of a system in accordance with one ormore embodiments of the invention. As shown in FIG. 1, the systemincludes a host (100), a network interface card (NIC) (105), multiplevirtual NICs (e.g., virtual NIC 1 (135), virtual NIC 2 (140), virtualNIC 3 (145)), a memory allocator (130), multiple virtual network stacks(VNSs) (e.g., VNS 1 (162), VNS 2 (164), VNS 3 (166)), and multiplepacket destinations (e.g., packet destination 1 (170), packetdestination 2 (175), packet destination 3 (180)). Each of thesecomponents is described below.

The NIC (105) provides an interface between the host (100) and a network(not shown) (e.g., a local area network, a wide area network, a wirelessnetwork, etc.). More specifically, the NIC (105) includes a networkinterface (NI) (i.e., the hardware on the NIC used to interface with thenetwork). For example, the NI may correspond to an RJ-45 connector, awireless antenna, etc. The packets received by the NI are then forwardedto other components on the NIC (105) for processing. In one embodimentof the invention, the NIC (105) includes one or more receive rings(e.g., receive ring 1 (115), receive ring 2 (120), receive ring 3(125)). In one embodiment of the invention, the receive rings (e.g.,receive ring 1 (115), receive ring 2 (120), receive ring 3 (125))correspond to portions of memory within the NIC (105) used totemporarily store packets received from the network. Further, in oneembodiment of the invention, a ring element of the receive rings (e.g.,receive ring 1 (115), receive ring 2 (120), receive ring 3 (125)) maypoint to host memory. In one embodiment of the invention, the classifier(110) is configured to analyze the incoming network traffic, typicallyin the form of packets, received from the network (not shown).

In one embodiment of the invention, analyzing individual packetsincludes determining to which of the receive rings (e.g., receive ring 1(115), receive ring 2 (120), receive ring 3 (125)) each packet isforwarded. In one embodiment of the invention, analyzing the packets bythe classifier (110) includes analyzing one or more fields in each ofthe packets to determine to which of the receive rings (e.g., receivering 1 (115), receive ring 2 (120), receive ring 3 (125)) the packetsare forwarded. As an alternative, the classifier (110) may use thecontents of one or more fields in each packet as an index into a datastructure that includes information necessary to determine to whichreceive ring (e.g., receive ring 1 (115), receive ring 2 (120), receivering 3 (125)) that packet is forwarded. The classifier (110) may alsouse other data found in the packet, such as the destination Media AccessControl (MAC) address, to classify the packet.

In one or more embodiments of the invention, the classifier (110) placespackets into receive rings (e.g., receive ring 1 (115), receive ring 2(120), receive ring 3 (125)) based on the packet destination (e.g.,packet destination 1 (170), packet destination 2 (175), packetdestination 3 (180)). As a result, each packet is linked to a packetdestination (e.g., packet destination 1 (170), packet destination 2(175), packet destination 3 (180)) even before the packet enters thehost (100). The classifier (110) may be implemented by a separatemicroprocessor embedded on the NIC (105). Alternatively, the classifier(110) may be implemented in software stored in memory (e.g., firmware,etc.) on the NIC (105) and executed by a microprocessor on the NIC(105).

In one or more embodiments of the invention, the host (100) may includea device driver (not shown) and one or more virtual NICs (e.g., virtualNIC 1 (135), virtual NIC 2 (140), virtual NIC 3 (145)). In oneembodiment of the invention, the device driver provides an interfacebetween the receive rings (e.g., receive ring 1 (115), receive ring 2(120), receive ring 3 (125)) and the host (100). More specifically, thedevice driver (not shown) exposes the receive rings (e.g., receive ring1 (115), receive ring 2 (120), receive ring 3 (125)) to the host (100).In one embodiment of the invention, each of the virtual NICs (e.g.,virtual NIC 1 (135), virtual NIC 2 (140), virtual NIC 3 (145)) isassociated with one or more receive rings (e.g., receive ring 1 (115),receive ring 2 (120), receive ring 3 (125)). In other words, a virtualNIC (e.g., virtual NIC 1 (135), virtual NIC 2 (140), virtual NIC 3(145)) receives incoming packets from a corresponding receive ring(s)(e.g., receive ring 1 (115), receive ring 2 (120), receive ring 3(125)). In one or more embodiments of the invention, outgoing packetsare forwarded from a virtual NIC (e.g., virtual NIC 1 (135), virtual NIC2 (140), virtual NIC 3 (145)) to a corresponding transmit ring (notshown), which temporarily stores the packet before transmitting thepacket over the network. In one or more embodiments of the invention,receive rings (e.g., virtual NIC 1 (135), virtual NIC 2 (140), virtualNIC 3 (145)) and transmit rings (not shown) are implemented as ringbuffers in the NIC (105).

In one or more embodiments of the invention, the virtual NICs (e.g.,virtual NIC 1 (135), virtual NIC 2 (140), virtual NIC 3 (145)) areoperatively connected to packet destinations (e.g., packet destination 1(170), packet destination 2 (175), packet destination 3 (180)), whichinclude containers and applications, via VNSs (e.g., VNS 1 (162), VNS 2(164), VNS 3 (166)). The virtual NICs (e.g., virtual NIC 1 (135),virtual NIC 2 (140), virtual NIC 3 (145)) provide an abstraction layerbetween the NIC (105) and the packet destinations (e.g., packetdestination 1 (170), packet destination 2 (175), packet destination 3(180)) on the host (100). More specifically, each virtual NIC (e.g.,virtual NIC 1 (135), virtual NIC 2 (140), virtual NIC 3 (145)) operateslike a NIC (105). For example, in one embodiment of the invention, eachvirtual NIC (e.g., virtual NIC 1 (135), virtual NIC 2 (140), virtual NIC3 (145)) is associated with one or more IP addresses, associated withone or more MAC addresses, associated with one or more ports, andconfigured to handle one or more protocol types. Thus, while the host(100) may be operatively connected to a single NIC (105), packetdestinations (e.g., packet destination 1 (170), packet destinations 2(175), packet destinations 3 (180)) executing on the host (100) operateas if the host (100) is bound to multiple NICs.

In one embodiment of the invention, each VNS (e.g., VNS 1 (162), VNS 2(164), VNS 3 (166)) includes functionality to process packets inaccordance with various protocols used to send and receive packets(e.g., Transmission Communication Protocol (TCP), Internet Protocol(IP), User Datagram Protocol (UDP), etc.). Further, each VNS (e.g., VNS1 (162), VNS 2 (164), VNS 3 (166)) may also include functionality, asneeded, to perform additional processing on the incoming and outgoingpackets. This additional processing may include, but is not limited to,cryptographic processing, firewall routing, etc.

In one or more embodiments of the invention, the VNSs (e.g., VNS 1(162), VNS 2 (164), VNS 3 (166)) correspond to network stacks withnetwork layer and transport layer functionality. In one embodiment ofthe invention, network layer functionality corresponds to functionalityto manage packet addressing and delivery on a network (e.g.,functionality to support IP, Address Resolution Protocol (ARP), InternetControl Message Protocol, etc.). In one embodiment of the invention,transport layer functionality corresponds to functionality to manage thetransfer of packets on the network (e.g., functionality to support TCP,UDP, Stream Control Transmission Protocol (SCTP), etc.). In one or moreembodiments of the invention, the VNSs (e.g., VNS 1 (162), VNS 2 (164),VNS 3 (166)) implement an IP layer (not shown) and a TCP layer (notshown).

In one embodiment of the invention, the virtual NIC (e.g., virtual NIC 1(135), virtual NIC 2 (140), virtual NIC 3 (145)) may be bound to avirtual machine (e.g., Xen Domain) instead of a VNS (e.g., VNS 1 (162),VNS 2 (164), VNS 3 (166)). In such cases, the virtual NIC (e.g., virtualNIC 1 (135), virtual NIC 2 (140), virtual NIC 3 (145)) is bound to aninterface (e.g., a Xen interface), where the interface enables thevirtual NIC (e.g., virtual NIC 1 (135), virtual NIC 2 (140), virtual NIC3 (145)) to communicate to with the virtual machine. In one embodimentof the invention, the aforementioned virtual machine includes its ownVNS and includes its own operating system (OS) instance, which may bedifferent than the OS executing on the host.

During initialization, the packet destination (or the virtual machine)is linked to a processor (i.e., a CPU) and associated with a VNS (e.g.,VNS 1 (162), VNS 2 (164), VNS 3 (166)) (or interface). In one or moreembodiments of the invention, assigning a packet destination to a VNS(e.g., VNS 1 (162), VNS 2 (164), VNS 3 (166)) involves assigning thepacket destination an unique identifier (ID) or assigning a VNS (e.g.,VNS 1 (162), VNS 2 (164), VNS 3 (166)) a unique ID, where the VNS (e.g.,VNS 1 (162), VNS 2 (164), VNS 3 (166)) is associated with the packetdestination. In the case of the virtual machine, the virtual machine orthe interface associated with the virtual machine is assigned a uniqueID. Alternatively, a virtual NIC (e.g., virtual NIC 1 (135), virtual NIC2 (140), virtual NIC 3 (145)) associated with a VNS (or interface of avirtual machine) may be assigned a unique ID that may be used toidentify the VNS (or the interface of the virtual machine.

The memory allocator (130) is responsible for allocating virtual memoryto the packet destinations (e.g., packet destination 1 (170), packetdestination 2 (175), packet destination 3 (180)) (or virtual machines).In one or more embodiments of the invention, virtual memory is allocatedfrom a virtual memory pool (i.e., the total virtual memory which thehost (100) or processes executing thereon may use).

In one embodiment of the invention, a packet destination (or virtualmachine) uses the allocated virtual memory to send and receive packets.In one or more embodiments of the invention, the virtual memoryallocated to a given packet destination (or virtual machine) is used tostore incoming and outgoing packets in transit between a receive ring ortransmit ring and the corresponding packet destination (or virtualmachine).

If a packet destination (or virtual machine) attempts to use virtualmemory in excess of the amount of virtual memory allocated to the packetdestination (or virtual machine), the memory allocator (130) (or aprocess associated with the memory allocator) blocks the packetdestination (or virtual machine) until the packet destination (orvirtual machine) has sufficient free virtual memory to issue additionaloutgoing packets or obtain incoming packets from the correspondingreceive ring. Because each receive ring is associated with a packetdestination (or virtual machine), the memory allocator (130) is able toenforce the virtual memory usage by keeping packets in the receive ringsuntil the packet destination (or virtual machine) has sufficientavailable virtual memory to process the packets. Similarly, a packetdestination (or virtual machine) is unable to transmit packets to theNIC (105) unless the packet destination (or virtual machine) hassufficient available virtual memory to store the packets.

As discussed above, the packet destination (e.g., packet destination 1(170), packet destination 2 (175), packet destination 3 (180)) (orvirtual machine) or the associated VNS (e.g., VNS 1 (162), VNS 2 (164),VNS 3 (166)) (or interface) is assigned a unique ID. Using theaforementioned ID, virtual memory usage for each packet destination (orvirtual machine) may be monitored and requests for virtual memory deniedwhen the packet destination (or virtual machine) attempts to obtainvirtual memory in excess of the virtual memory allocated to the packetdestination (or virtual machine).

In one embodiment of the invention, if a packet destination (or virtualmachine) attempts to obtain virtual memory in excess of the virtualmemory allocated to the packet destination (or virtual machine), thenthe memory allocator may re-allocate the virtual memory based on thepriority associated with the packet destination (or virtual machine)that is attempting to obtain virtual memory in excess of the allocatedvirtual memory. In such cases, a packet destination (or virtual machine)with a higher priority may be allocated additional virtual memory at theexpense of a packet destination (or virtual machine) with a lowerpriority.

FIG. 2 shows a schematic diagram of a memory allocation system inaccordance with one or more embodiments of the invention. The memoryallocation system includes a memory allocator (130), virtual memory(200), multiple slabs associated with central processing units (CPUs)(e.g., CPU 1 slab (205), CPU 2 slab (210)), and allocated virtual memoryfor each virtual NIC (e.g., VNIC 1 memory (215), VNIC 2 memory (220),VNIC 2 memory (225), VNIC 3 memory (230)).

Virtual memory (200) in FIG. 2 denotes the virtual memory pool for thehost (100 in FIG. 1). Further, the virtual memory (200), as shown inFIG. 2, is divided into smaller portions (i.e., CPU slabs (205, 210)),where each of the portions is associated with a CPU executing on thehost (100 in FIG. 1).

In one or more embodiments of the invention, the virtual memory (200)may be located on multiple physical memory devices, CPU caches, mainmemory (e.g., dynamic Random Access Memory (RAM)), and disk storage. Inaddition, contiguous regions of virtual memory may actually be locatedon noncontiguous regions of physical memory on multiple physical memorydevices. In one or more embodiments of the invention, pages of a powerof 2 length (e.g., 1024 bytes-8192 bytes) are mapped to page frames, orcontiguous regions of physical memory. In one or more embodiments of theinvention, a memory management unit (MMU) (not shown) within the memoryallocator (130) maps a contiguous region of virtual memory (200) to pageframes scattered around physical memory.

In one or more embodiments of the invention, a CPU slab (e.g., CPU 1slab (205), CPU 2 slab (210)) is made up of one or more page framesallocated from virtual memory (200). As discussed above, a CPU slab(e.g., CPU 1 slab (205), CPU 2 slab (210)) is allocated for a CPU. Forexample, if a system contains five CPUs, then memory allocator (130)allocates one CPU slab (e.g., CPU 1 slab (205), CPU 2 slab (210)) foreach CPU from virtual memory (200) for a total of five CPU slabs.

In addition, each packet destination (or virtual machine) that uses aCPU is allocated virtual memory within that CPU slab (e.g., CPU 1 slab(205), CPU 2 slab (210)). In one or more embodiments of the invention,virtual memory within each CPU slab is allocated on a packet destination(or virtual machine) basis and, alternatively, as shown in FIG. 2, on avirtual NIC basis.

As discussed above, each packet destination (or virtual machine) may beassociated with an ID or alternatively, each virtual NIC may beassociated with an ID. Those skilled in the art will appreciate thatallocating virtual memory based on either virtual NICs or packetdestination (or virtual machines) will produce the same result sincepacket destinations (or virtual machines) are mapped one-to-one tovirtual NICs.

Continuing with the discussion of FIG. 2, each CPU slab (e.g., CPU 1slab (205), CPU 2 slab (210)) may include virtual memory for eachvirtual NIC. In one or more embodiments of the invention, a virtual NICis allocated virtual memory (e.g., VNIC virtual memory (215, 220, 225,230)) in a CPU slab (e.g., CPU 1 slab (205), CPU 2 slab (210)) for eachCPU the virtual NIC (or packet destination or virtual machine) uses.

As shown in FIG. 2, virtual NIC 1 is allocated virtual memory (215) inthe CPU 1 slab (205) and virtual NIC 3 is allocated virtual memory (230)in the CPU 2 slab, indicating that virtual NIC 1 uses CPU 1 and virtualNIC 3 uses CPU 2. However, virtual NIC 2 is allocated virtual memory(e.g., VNIC 2 virtual memory (220), VNIC 2 virtual memory (225)) in boththe CPU 1 slab (205) and the CPU 2 slab (210), which indicates thatvirtual NIC 2 uses both CPU 1 and CPU 2. In one or more embodiments ofthe invention, the virtual memory is allocated from one or more pages invirtual memory (200). Further, the virtual memory allocated for eachvirtual NIC may include one or more pages in virtual memory (200), or apage may be shared by multiple virtual NICs, depending on the page size.For example, a virtual NIC that is given 1024 bytes of memory may beallocated one page of 1024 bytes or share a larger page (e.g., 2048bytes or 4096 bytes) with other virtual NICs.

In one or more embodiments of the invention, the virtual memoryallocated for each virtual NIC is allocated based on priority. Forexample, if virtual NIC 1 has a higher priority than virtual NIC 2, thenthe memory allocator (130) will allocate more virtual memory to VNIC 1than to VNIC 2. In addition, if virtual NIC 1 experiences a memoryshortage, the memory allocator (130) may reallocate virtual memory fromVNIC 2 to VNIC 1.

FIG. 3 shows a flow diagram in accordance with one or more embodimentsof the invention. More specifically, FIG. 3 shows a method for enforcingvirtual memory usage by a packet destination (or virtual machine) inaccordance with one embodiment of the invention.

Initially, a packet is received by a NIC (Step 301). The packet isclassified (Step 303) by the classifier and placed into the appropriatereceive ring (Step 305). In one or more embodiments of the invention,the classifier is configured to differentiate packets by packetdestination (or virtual machine) and place each packet into a receivering corresponding to the packet destination (or virtual machine).

At this stage, the ID associated with the packet destination (or virtualmachine) is obtained (Step 307). In one embodiment of the invention, theID is directly associated with the packet destination (or virtualmachine). Alternatively, the ID may be associated with a virtual NIC orVNS associated with the packet destination. In addition, the ID may beassociated with a virtual NIC or an interface associated with thevirtual machine. Because each receive ring is associated with a virtualmachine or packet destination, the aforementioned ID may be obtainedbased on the receive ring to which the packet is forwarded.Alternatively, the ID may be associated with the receive ring, such thatwhen a packet is received by the receive ring, the ID is associated withthe packet.

Using the ID, a determination is made about whether virtual memoryassociated with the packet destination (or virtual machine) is available(Step 309). In one embodiment of the invention, the determinationincludes sending a request for virtual memory to the memory allocator(or a process related to the memory allocator) where the requestincludes the amount of virtual memory required and the ID.

Upon receiving the request, the memory allocator (or a process relatedto the memory allocator) uses the ID to determine the amount ofavailable virtual memory for the packet destination (or virtual machine)associated with the ID. In one embodiment of the invention, the memoryallocator (or a process related to the memory allocator) includesinformation about the total amount of virtual memory allocated to agiven packet destination (or virtual machine) indexed by ID as well asthe amount of the aforementioned allocated virtual memory currentlybeing used by the given packet destination (or virtual machine).

Using this information, the memory allocator (or a process related tothe memory allocator) determines whether there is sufficient virtualmemory available to service the request. Said another way, the memoryallocator (or a process related to the memory allocator) determineswhether the sum of the virtual memory currently being used by the packetdestination (or virtual machine) plus the requested virtual memory willexceed the total amount of virtual memory allocated to the packetdestination (or virtual machine).

If virtual memory is available (i.e., the sum of the virtual memorycurrently being used by the packet destination (or virtual machine) plusthe requested virtual memory does not exceed the total amount of virtualmemory allocated to the packet destination (or virtual machine)), thenthe packet is written to virtual memory (Step 311) and enters thesystem. Writing to virtual memory includes allocating virtual memory,writing the packet to the virtual memory, and updating the memoryallocator (or a process related to the memory allocator) to reflect thatpacket destination (or virtual machine) has allocated the additionalvirtual memory.

Alternatively, if virtual memory is not available, the packet stays inthe receive ring until sufficient virtual memory is available (Step 313)before being written to network memory (Step 311).

Outgoing packets from the packet destination or virtual machine arehandled similarly (i.e., using a similar process to the one described inFIG. 3). Specifically, each packet is stored in the packet destination(or virtual machine)until sufficient virtual memory is available; thepacket is then written to virtual memory and passed through the VNS (orinterface) to the associated virtual NIC. The virtual NIC, upon receiptof the packet, sends the packet to a transmit ring on the NIC, where thepacket is transmitted over the network.

FIG. 4 shows a flow diagram in accordance with one or more embodimentsof the invention. More specifically, FIG. 4 shows a method for settingup the system in accordance with one embodiment of the invention.

Initially, a packet destination (or virtual machine) is created (Step401). Next, a VNS (or an interface) is created (Step 403). The VNS (orinterface) is associated to the packet destination (or virtual machine)(Step 405).

A virtual NIC is subsequently created and associated with the VNS (orinterface) (Step 407). The ID associated with one of the packetdestination (or virtual machine), the VNS (or interface), or virtual NICis then sent to the memory allocator, where the ID is registered (Step409). The aforementioned ID is set during the creation of the component(i.e., the packet destination (or virtual machine), the VNS (orinterface), or the virtual NIC) that includes the ID.

Returning to the discussion of FIG. 4, virtual memory is then allocatedto the packet destination (or virtual machine) (Step 411). Because theID is registered with the memory allocator, the ID may be subsequentlyused (as discussed in FIG. 3) to allocate virtual memory for incomingand outgoing packets and to enforce virtual memory usage by packetdestinations (or virtual machines). Thought not shown in FIG. 4, the IDmay be associated with a receive ring associated with the packetdestination (or virtual machine), such that each time a packet is placedin the receive ring, the packet is associated with the ID.

The invention may be implemented on virtually any type of computerregardless of the platform being used. For example, as shown in FIG. 5,a computer system (500) includes a processor (502), associated memory(504), a storage device (506), and numerous other elements andfunctionalities typical of today's computers (not shown). The computer(500) may also include input means, such as a keyboard (508) and a mouse(510), and output means, such as a monitor (512). The computer system(500) is connected to a local area network (LAN) or a wide area network(e.g., the Internet) (not shown) via a network interface connection (notshown). Those skilled in the art will appreciate that these input andoutput means may take other forms.

Further, those skilled in the art will appreciate that one or moreelements of the aforementioned computer system (500) may be located at aremote location and connected to the other elements over a network.Further, the invention may be implemented on a distributed system havinga plurality of nodes, where each portion of the invention (e.g., packetdestination or virtual machine, CPU, or virtual memory) may be locatedon a different node within the distributed system. In one embodiment ofthe invention, the node corresponds to a computer system. Alternatively,the node may correspond to a processor with associated physical memory.The node may alternatively correspond to a processor with shared memoryand/or resources. Further, software instructions to perform embodimentsof the invention may be stored on a computer readable medium such as acompact disc (CD), a diskette, a tape, a file, or any other computerreadable storage device.

While the invention has been described with respect to a limited numberof embodiments, those skilled in the art, having benefit of thisdisclosure, will appreciate that other embodiments can be devised whichdo not depart from the scope of the invention as disclosed herein.Accordingly, the scope of the invention should be limited only by theattached claims.

1. A method for processing a packet, comprising: receiving a packet fora first target on a host; classifying the packet; sending the packet toa receive ring based on the classification; obtaining an identifier (ID)based on the classification, wherein the ID is associated with the firsttarget; sending a request for virtual memory on the host, wherein therequest comprises the ID; determining, using the ID, whether the firsttarget has exceeded a virtual memory allocation associated with thefirst target; if the first target does not exceed the virtual memoryallocation: allocating the virtual memory; storing the packet in thevirtual memory on the host; and updating the virtual memory allocationassociated with the first target to reflect the allocation of thevirtual memory; and if the first target exceeds the virtual memoryallocation: waiting until the first target is not exceeding the virtualmemory allocation.
 2. The method of claim 1, wherein the ID is stored inthe receive ring.
 3. The method of claim 1, wherein the first target isone selected from a group consisting of a packet destination and avirtual machine.
 4. The method of claim 1, wherein the virtual memoryallocation associated with the first target is stored in a memoryallocator on the host.
 5. The method of claim 1, wherein the virtualmemory allocation associated with the first target is set when the firsttarget is created.
 6. The method of claim 1, wherein the virtual memoryallocation associated with the first target is decreased if a secondtarget requires additional virtual memory and the second target isassociated with a higher priority than the first target.
 7. The methodof claim 1, wherein the virtual memory allocation comprises a portion ofa virtual memory slab associated with a first processor and a portion ofa virtual memory slab associated with a second processor.
 8. The methodof claim 1, wherein the packet is sent to one selected from a groupconsisting of a virtual network stack and an interface after the packethas been stored in virtual memory on the host.
 9. A system, comprising:a network interface card (NIC) configured to: receive a packet for afirst target on a host; classify the packet; send the packet to areceive ring based on the classification; obtain an ID based on theclassification, wherein the ID is associated with the first target; andsend a request for virtual memory on the host, wherein the requestcomprises the ID; and the host, operatively connected to the NIC,configured to: determine, using the ID, whether the first target hasexceeded a virtual memory allocation associated with the first target;if the first target does not exceed the virtual memory allocation:allocate the virtual memory; obtain the packet from the receive ring;store the packet in the virtual memory on the host; and update thevirtual memory allocation associated with the first target to reflectthe allocation of the virtual memory; and if the first target exceedsthe virtual memory allocation: wait until the first target is notexceeding the virtual memory allocation.
 10. The system of claim 9,wherein the ID is stored in the receive ring.
 11. The system of claim 9,wherein the first target is one selected from a group consisting of apacket destination and a virtual machine.
 12. The system of claim 9,wherein the virtual memory allocation associated with the first targetis stored in a memory allocator on the host.
 13. The system method ofclaim 9, wherein the virtual memory allocation associated with the firsttarget is set when the first target is created.
 14. The system method ofclaim 9, wherein the virtual memory allocation associated with the firsttarget is decreased if a second target requires additional virtualmemory and the second target is associated with a higher priority thanthe first target.
 15. The system method of claim 9, wherein the virtualmemory allocation comprises a portion of a virtual memory slabassociated with a first processor and a portion of a virtual memory slabassociated with a second processor.
 16. The system method of claim 9,wherein the packet is sent to one selected from a group consisting of avirtual network stack and an interface after the packet has been storedin virtual memory on the host.
 17. A computer readable medium comprisinginstructions for a method for processing a packet, the methodcomprising: receiving a packet for a first target on a host; classifyingthe packet; sending the packet to a receive ring based on theclassification; obtaining an ID based on the classification, wherein theID is associated with the first target; sending a request for virtualmemory on the host, wherein the request comprises the ID; determining,using the ID, whether the first target has exceeded a virtual memoryallocation associated with the first target; if the first target doesnot exceed the virtual memory allocation: allocating the virtual memory;storing the packet in the virtual memory on the host; and updating thevirtual memory allocation associated with the first target to reflectthe allocation of the virtual memory; and if the first target exceedsthe virtual memory allocation: waiting until the first target is notexceeding the virtual memory allocation.
 18. The computer readablemedium of claim 17, wherein the ID is stored in the receive ring. 19.The computer readable medium of claim 17, wherein the first target isone selected from a group consisting of a packet destination and avirtual machine.
 20. The computer readable medium of claim 17, whereinthe virtual memory allocation associated with the first target isdecreased if a second target requires additional virtual memory and thesecond target is associated with a higher priority than the firsttarget.